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Abstract 

Background: Well-designed and properly executed randomized controlled trials (RCTs) provide the best evidence 
on the efficacy of healthcare interventions. Mental health has a strong tradition of using trial to evaluate 
treatments, but the translation of research to clinical practice is not always easy. Even well-conducted trials do not 
necessarily address the needs of every day care and trials can reflect local needs and the specific culture in which 
they are undertaken. Generalizing results to other contexts can become problematic but these trials may, 
nevertheless, be very helpful within their own context. Moreover, pathways for drug approval can be different 
depending on local regulatory agencies. Local trials are helpful for decision-making in the region from which they 
come, but should not be viewed in isolation. National quantity and quality of trials may vary across nations. 
The aim of this study is to quantify trialing activity in Italy from 1948 until 2009 and to describe characteristics of 
these trials. In addition, we evaluated change over time in three keys aspects: sample size, follow-up duration, and 
number of outcomes. 

Methods: We used the Cochrane Schizophrenia Group's register that contains 16,000 citations to 13,000 studies 
relating only to people with schizophrenia or schizophrenia-like illness. Randomized controlled trials and controlled 
clinical trials undertaken in Italy and involving pharmacological interventions were included. 

Results: The original search identified 155 records of potentially eligible studies, 74 of which were excluded 
because do not meet inclusion criteria. A total of 81 studies were included in the analysis. The majority of trials 
were conducted in north Italy, and published in international journals between 1981 and 1995. The majority of 
studies (52 out of 81) used standardized diagnostic criteria for schizophrenia disorder. They were defined as 
randomized and used blind methods to administer treatment. However, most failed to report detail regarding 
methodological procedures and it is difficult to ascertain which studies are associated with a low risk of bias. 

Conclusions: Trials should be designed to address the needs of everyday care with the aim of following large 
samples of typical patients in the long term. The Italian tradition in the area of trialing treatments for people with 
schizophrenia is not as strong as in many other similar countries and Italy should be producing more, better, 
independent, and clinically relevant trials. 
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Background Streptomycin randomized trial was published and 

Good mental health research is critical to guide health- remains a landmark of modern healthcare evaluation 
care professionals to make informed decisions about the [3]. After that many specialties began to adopt randomi- 
effects of most interventions. Early examples do exist of zation for the evaluation of treatments, 
attempts at fair evaluation of healthcare, including sev- Mental health has a strong tradition of using trials [4] , 
eral from Italy [1,2]. However, in 1948 the UK MRC but translation of research to clinical practice is not 

always easy. For patients, carers, and policymakers, local 
_ data are important. Even well-conducted trials, if under- 
taken in a very dissimilar care-culture may be difficult 
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whether they concur - or contradict - other similar stu- 
dies from afar. The totality of evidence must be consid- 
ered but the local perspective not ignored. Not to have 
local studies leaves all interested in the effects of care 
vulnerable to importing data of limited applicability. 
Some nations, however, have produced few trials. For 
schizophrenia trials, measures of national wealth and 
not public health burden loosely predict the research 
activity of a country [5]. Italy, however, is the 10th most 
wealthy nation in terms of GDP http://www.nationmas- 
ter.com/ graph/ eco_gdp-economy-gdp. 

The quality of mental health research, and trials in 
particular, may also vary across nations [4]. For example, 
the volume and quality of trial research from China has 
been considered in many surveys and quality remains a 
major concern [6-10]. Elsewhere it has been shown that 
pioneering mental health trials from low and middle 
income countries are of as mixed quality as their more 
accessible counterparts from richer nations but cannot 
be identified in commonly used bibliographic databases 
[11]. Nearer home, in Europe, Romania's mental health 
research has been the focus of recent investigation and 
the increasing dominance of pharmaceutical industry 
noted [12]. 

The Cochrane Schizophrenia Group produces and 
maintains a register of all studies http://szg. Cochrane, 
org/cochrane-schizophrenia-group-specialised-register. 
This involves regular and systematic searching of 71 
databases. The studies identified in this way are reliably 
indexed by country. The aim of this study is to use part 
of this dataset to quantify trialing activity in one area of 
mental healthcare over time in Italy and describe char- 
acteristics of these trials. Specifically, we evaluated con- 
tent and risk of bias of Italian trials relevant to people 
with schizophrenia from 1948 until 2009 for three key 
methodological aspects: sample size, duration, and num- 
ber of outcomes. 

Methods 

Source 

We searched the Cochrane Schizophrenia Group Register 
without time limitation. The search was made in May 
2011 and the last version of register was updated in April 
2011. The register includes all published and unpublished 
references to randomized, quasi-randomized and con- 
trolled clinical trials without language restrictions. The 
register is maintained on Meerkat 1.6. The Cochrane 
Schizophrenia Group's register contains 16,000 citations 
to 13,000 studies relating only to people with schizophre- 
nia or schizophrenia-like illness http://szg.cochrane.org/ 
cochrane-schizophrenia-group-specialised-register. These 
studies are reliably indexed regarding the country of ori- 
gin, the interventions under study, and the number of 
participants. 



Types of studies 

We included all randomized controlled trials and con- 
trolled clinical trials undertaken in Italy. Studies were 
included if any pharmacological treatment was compared 
with other active pharmacological treatments or placebo. 
Only studies that enrolled patients in Italy were consid- 
ered. Multicenter studies were included if all centers 
enrolling patients were located in Italy. 

Selection of trials and data extraction 

From the Cochrane Schizophrenia Group Register we 
extracted all records corresponding to studies carried out 
in Italy. We examined all titles and abstracts, and 
obtained full texts if the word 'random' or 'randomized' 
or 'control' or 'controlled' was present in the title and/or 
abstract. MP read the full texts, determined whether they 
met inclusion criteria, and extracted the data. Data were 
extracted using an electronic spreadsheet. Considerable 
care was taken to exclude duplicate publications. In 
order to ensure consistency, CB and CEA carried out a 
reliability check on all data extracted by comparing the 
data abstracted in the electronic spreadsheet with the 
paper version of each study. In case of disagreement 
between reviewers this was resolved by discussion. 

We considered the following variables: year of publica- 
tion, geographic area (north, center, south of Italy), lan- 
guage of publication (English, Italian), pharmacological 
treatment (Antipsychotics according to the WHO Anato- 
mical Therapeutic Chemical [ATC] classification system, 
other pharmacological treatments), sample size, weeks of 
follow-up, diagnostic criteria (standardized criteria such 
as DSM or ICD, implicit criteria such as clinical judg- 
ment, or unclear criteria), description of random alloca- 
tion and blinding (using the Cochrane Collaboration 'Risk 
of BIAS' tool criteria), number of outcome measures 
(number of different psychopathological dimensions ana- 
lyzed, as indexed in the specialized register), and use of 
the CONSORT flow-diagram. 

Data presentation 

We calculated simple percentages (%) with 95% confi- 
dence intervals (CI). To ascertain whether sample size, 
length of follow-up, and number of efficacy measures have 
increased in the last 45 years, we used a box plot diagram 
and a non-parametric test for trend (extension of the Wil- 
coxon rank-sum test). Spearman's rank correlation coeffi- 
cients for all pairs of variables were additionally calculated 
using data in continuous format. STATA 11 was used to 
carry out the statistical analysis. 

Results 

General characteristics 

The original search identified 155 records of potentially 
eligible studies, 74 of which did not meet inclusion 
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criteria (Figure 1). For the remaining 81 records full text 
was retrieved and data extracted. The characteristics of 
included studies are presented in Table 1. 

Most studies (52% CI 40-63) were conducted in north 
Italy, the majority were published between 1981 and 
1995 (46% CI 34-57), and 72% (CI 60-81) were in Eng- 
lish. The majority of studies evaluated antipsychotic 
drugs (79%, CI 69-87), while a minority assessed the 
beneficial effect of other drug treatments. Most also 
used standardized diagnostic criteria for defining schizo- 
phrenia (64% CI 53-75), whereas 32% (CI 22-43) used 
implicit criteria and 4% (CI 1-10) do not specify any 
diagnostic criteria. 



Methodological characteristics 

Most trials were of short duration, with only 20 (24.6% 
CI 15.7-35.5) being of medium or long-term follow-up 
(13 weeks or more) (Table 2). In 10 cases the length of 
follow-up was unclear. Duration does not increase over 
time (z for trend = -0.41, P = 0.685) (Figure 2). Spear- 
man's rank correlation coefficient confirmed no associa- 
tion between year and length of follow-up (rho = -0.030; 
P = 0.814) 

Two-thirds of trials had less than 40 participants (67% 
CI 55-77); sample size only minimally increased over 
time (Figure 3), as shown by Spearman's rank correla- 
tion coefficient (rho = 0.259; P = 0.020) but not by test 
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Figure 1 PRISMA flow diagram 
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Table 1 Characteristics of schizophrenia drug trials 
published in Italy from 1948 to 2009 (n = 81 studies) 



(n) 



% (95% CI) 



Area of Italy 

North 42 51.8(40.4-63.0) 

Center 19 23.4 (14.7-34.1) 

South 20 24.6 (15.7-35.5) 
Year of publication* 

1948-1980 20 25.3 (16.2-36.3) 

1981-1995 36 45.5 (34.3-57.1) 

1996-2009 23 29.1 (19.4-40.4) 
Language of publication 

English 58 71.6 (60.4-81.0) 

Italian 23 28.4 (18.9-39.5) 
Drug treatment 

Antipsychotics 64 79.0 (68.5-87.2) 

Other drugs 17 20.1 (12.7-31.4) 
Diagnostic criteria 

Unclear 3 3.7 (0.77-10.4) 

Standardized criteria 52 64.2 (52.7-74.5) 

Implicit criteria 26 32.1 (22.1-43.3) 



*Two unpublished studies not included 
CI confidence interval. 



Table 2 Main methodological characteristics of 
schizophrenia drug trials published in Italy from 1948 to 
2009 (n = 81 studies) 





n 


% (95% CI) 


Sample size 






Min-20 


22 


27.1 (17.8-38.1) 


21-40 


32 


39.5 (28.8-50.9) 


40-max 


27 


33.3 (23.2-44.6) 


Length of follow-up (weeks) 






Unclear 


10 


12.3 (6.08-21.5) 


2--1 


30 


24.6 (15.7-35.5) 


5-12 


21 


25.9 (16.8-36.8) 


13+ 


20 


24.6 (15.7-35.5) 


Randomization 






Randomized, no details 


51 


62.9 (51.5-73.4) 


Randomized with details 


7 


8.6 (3.54-16.9) 


Unclear 


23 


28.4 (18.9-39.5) 


Blinding 






Single blind 


6 


7.4 (2.76-15.4) 


Double blind 


47 


58.0 (46.5-68.9) 


Unclear 


28 


34.5 (24.3-45.9) 


Outcome measures (n) 






1-5 


33 


40.7 (29.9-52.2) 


6-15 


35 


43.2 (32.2-54.6) 


16-60 


13 


1 6.0 (8.83-25.8) 


Patient flow diagram 






Yes 


2 


2.4 (0.30-8.63) 


No 


79 


97.6 (91.3-99.6) 


CI confidence interval 



for trend (z for trend = 0.48, P = 0.628). While the use 
of standardized diagnostic criteria progressively 
increased, accounting for more that 70% of trials pub- 
lished after 1995 (17/23), the use of implicit criteria 
dropped, accounting for less than 20% of trials pub- 
lished after 1995 (4/23). 

Most trials were described as 'randomized' even 
though only seven (9% CI 4-17) provided details about 
the methods of allocation (Table 2). In 23 studies (28% 
CI 19-40) the allocation procedure was unclear. Single 
blind was adopted in six studies (7% CI 3-15), double 
blind was adopted in 47 studies (58% CI 47-69). For the 
remaining 28 studies (35% CI 24-46) it was unclear if 
blinding was used or not. 

The number of outcomes measures ranged from 1 to 
5 in 33 studies (41%, CI 30-52), from 6 to 15 in 35 trials 
(43%, CI 32-55) and from 16 to 60 in 13 studies (16% 
CI 9-26). Numbers of outcomes measures increases over 
time (z for trend = 3.32, P = 0.001) (Figure 4). Spear- 
man's rank correlation coefficient confirmed the associa- 
tion between year and number of outcomes (rho = 
0.436; P < 0.001). 

The CONSORT statement was first published in 1996 
[13]. Two of the 23 studies since that time reported a 
patient flow diagram (8% CI 1-28). 

Discussion 

To our knowledge, this is the first survey investigating 
content and volume of trial activity in Italy. Although the 
majority of Italian schizophrenia trials were published in 
international journals, most failed to report basic metho- 
dological details such as, for example, information about 
the methods of random allocation, and its concealment 
from the study investigators, or how blinding was pre- 
served. It is therefore difficult to ascertain which studies 
are associated with a low risk of bias. This compelling 
issue similarly applies to trial activity conducted in other 
countries. China, for example, a country which has pro- 
gressively increased its production of randomized studies 
in the last 10 years, is still under scrutiny for some aspects 
of trial quality, including random allocation, blinding, and 
dropout reporting. This seems quite relevant in different 
areas of medicine [14], as well as in mental health [15]. 
Poor reporting in mental health is common, involves both 
high and low income countries [4], and leads to risk in 
interpreting results [16]. In most cases it is not possible to 
go beyond identifying studies that seem to be randomized 
trials. This further reinforces the need to develop tools to 
better describe and appraise the adequacy of the randomi- 
zation process in a culturally sensitive manner. Clearly, 
random allocation is a scientific process and it should not 
differ across cultures, but its reporting should take into 
consideration the different meaning that the world ran- 
dom might have in different cultures. 
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Figure 2 Length of Italian schizophrenia drug Trials published between 1984 and 2009 (n = 81) The horizontal line represents the 
median, the box extends to cover the interquartile range and the vertical line extends to the extremes. 



The Consolidated Standard of Reporting Trial (CON- 
SORT) statement [13,17] facilitates complete and trans- 
parent reporting, aids critical appraisal and interpretation 
of results. This simple checklist for reporting has been 



widely adopted [18]. Good trial reporting is important 
not only for guiding clinicians towards correct decision- 
making, but also for regulatory agencies working on drug 
approval. Certainly, clinicians and regulatory authorities 
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Figure 4 Number of outcome measures employed in Italian schizophrenia drug trials published between 1984 and 2009 (n = 81). The 

horizontal line represents the median, the box extends to cover the interquartile range and the vertical line extends to the extremes. 



should require higher standards from clinical trial reports 
[19]. Researchers and editors no longer have an excuse 
for bad reporting. 

In Italy, in this subspecialty, while sample size increased 
only minimally over time, patient selection criteria and 
outcome assessments have become much more sophisti- 
cated, as suggested by the increase in use of standardized 
diagnostic criteria and by the steadily increase in the num- 
ber of outcome measures. Although this trend may have 
increased the internal validity of findings, it has neverthe- 
less allowed study of only highly refined groups of people 
with schizophrenia. This increasing drift from real world 
practice makes it difficult to apply trial results to typical 
patients [20,21]. Similarly, we observed that the number of 
outcome measures has increased during the last 45 years. 
This confusion of measuring suggests, at the very least, a 
lack of consensus on what is important. Another example, 
if one was needed, supporting the need for a set of core 
outcome measures in this area [22] . Also increasing num- 
bers of measures inevitably enhanced the probability of 
detecting chance significant differences. Rarely had the 
trials addressed this issue in the statistical analysis or in 
the interpretation of findings. 

Our survey has some limitations. First, we did not 
include international multicenter studies where Italy was 
one site, possibly losing studies with good sample sizes 
and potentially better reporting. Second, as our focus 
was drug trials only, we collected no data on content 



and volume of trial activity in other fields of schizophre- 
nia treatment, including psychological treatments, psy- 
chosocial interventions, and organizational approaches. 
This limits the generalizability of our findings, as we do 
not know if the current standard of drug trials can be 
considered representative of the whole spectrum of trial 
activity. Finally, we focused our survey on some indica- 
tors only, while it would have been of interest to 
describe trial activity with respect to other aspects, such 
as for example setting, intention to treat versus per pro- 
tocol analysis, dropout reporting, economic support, 
ethics committee approval, and consent 'rituals'. 

Conclusions 

The Italian tradition in the area of evaluating treatments 
for people with schizophrenia is not as strong as in other 
similar countries [5]. Perhaps Italy has relied too much 
on studies undertaken outside its borders. There is an 
opportunity to produce more, better, independent, and 
nationally and internationally clinically relevant trials. 
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